WormShield: Fast Worm Signature Detection with Distributed Fingerprint Aggregation

Distributed Fingerprint Aggregation ∗ Min Cai, Kai Hwang, Jianping Pan, and Christos Papadopoulos Abstract: Fast detection of worm signatures is essential to contain zero-day worms using content filtering. Recent work has shown that it is feasible to detect worm signatures automatically by analyzing the repetition of worm substrings (i.e., fingerprints) and their address dispersion. However, at the early stage of a worm outbreak, individual edge networks are often short of enough worm samples to generate accurate signatures. In this paper, we present both theoretical modeling and experimental results on a collaborative worm signature detection system (WormShield) that employs distributed fingerprint filtering and aggregation in multiple edge networks. We discover the Zipf-like distributions of fingerprint repetition and address dispersion in reallife Internet traffic. Based on this property, distributed fingerprint filtering significantly reduces the aggregation traffic. WormShield monitors also utilize a novel distributed aggregation tree to aggregate the global information in a scalable and load-balancing fashion. We simulated CodeRed and Slammer worms on realistic Internet configurations with about 100K edge networks. On the average, our scheme using 256 collaborative monitors detects the signature of CodeRedI-v2 about 135 times faster than using equal number of isolated monitors. The signature detection speed of WormShield can be further improved by 19 times when 3, 000 collaborative monitors are deployed. In addition to speed gains, we observe less than 100 false signatures out of Internet traces with 23M packets. Each monitor generates roughly 6KB/s aggregation traffic, which is 0.03% of the 18MB/s link traffic sniffed. These results on speed, accuracy and overhead demonstrate the effectiveness and scalability of WormShield in fast worm signature detection.